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Abstract 


NASA is investigating external Visibility Systems (XVS) concepts which are a 
combination of sensor and display technologies designed to achieve an equivalent level 
of safety and performance to that provided by forward-facing windows in today’s 
subsonic aircraft. This report provides the background for conceptual XVS design 
standards for display and sensor resolution. 

XVS resolution requirements were derived from the basis of equivalent performance. 
Three measures were investigated: a) human vision performance ; b) see-and-avoid 
performance and safety; and c) see-to-follow performance. From these three factors, a 
minimum but perhaps not sufficient resolution requirement of 60 pixels per degree was 
shown for human vision equivalence. However, see-and-avoid and see-to-follow 
performance requirements are nearly double. 

Two areas are identified for additional investigation: a) the need to verify and validate 
these analytically-derived values by in-flight testing; and, b) the development of XVS 
display and sensor resolution requirements using modulation transfer function-based 
criteria as a measure of optical performance. 

This report also reviewed historical XVS testing. The trends in commercial video 
systems are very favorable such that emerging technology has the potential to meet the 
needs of an XVS system. Proof-of-concept testing has identified potential limitations. 
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1. Introduction 


A successful low boom supersonic aircraft design drives the shaping and configuration of 
the vehicle. One such conceptual configuration is shown in Figure 1 . As evident in this 
figure, the forward visibility for the flight crew is severely compromised as a result of the 
vehicle shaping. 

The National Aeronautics and Space Administration (NASA) is performing fundamental 
research, development, test and evaluation of flight deck and related technologies which 
may provide the required pilot visibility for these low-boom, supersonic configurations 
by use of an external Visibility System (XVS). XVS is a combination of sensor and 
display technologies designed to achieve an equivalent level of safety and performance to 
that provided by forward- facing windows in today’s aircraft. In this memorandum, 
conceptual standards for “equivalent performance and safety” are offered as a basis for 
the design of an XVS. Further, testing results for sensor and display resolution are 
briefly reviewed. 



Figure 1: Conceptual Low-Boom Supersonic Aircraft Configuration 

The XVS conceptual design standards are only notional and imply nothing more, since a 
formal basis for certification of an XVS has not yet been established by the Federal 
Aviation Administration (FAA) or other certification authority. However, these notional 
standards are offered using current FAA regulations and advisory material with support 
from previous NASA research and other related research areas. Using these notional 
standards, further research needs are identified by which to verify and validate these 
requirements. 

This report provides the background and summarization of notional XVS design 
standards for display and sensor resolution. Citations are used for textbooks and 
analytical and simulation models where more detailed factors associated with these 
requirements (e.g., visual acuity, sensor acuity and other factors) are necessary. This 
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report is intended to summarize the pacing requirements and technologies while 
identifying the research and technology development required to produce validated and 
verified XYS acuity requirements to support low-boom Supersonic Aircraft emerging 
within the 2015-2025 time frame. 

Two significant requirements which interact with the sensor and display resolution in an 
XVS design are not addressed in this work - Field-Of-View (FOV) and display 
collimation. These issues are discussed but standards are not provided. These should be 
addressed in forthcoming XVS research. 
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2. Background 

The Concorde and TU-144 supersonic aircraft used a mechanism which drooped the 
aircraft nose to enable pilot visibility from forward- facing windows for terminal 
operations including takeoff, landing, departure, arrival, and surface operations. For an 
aircraft of the class anticipated for the High Speed Civil Transport (HSCT) aircraft, the 
maximum takeoff weight of a fixed-nose configuration design using an XVS was 3.2% 
lower than that of a drooped-nose configuration. This substantial weight difference 
created a significant cost-benefit for the development of an XVS (Andrastek, 1999). 



Figure 2: TU-144 Drooped Nose Deployed 

Without an XVS or a drooped nose, the pilot of a low-boom supersonic aircraft cannot 
see in front of the aircraft by using their natural vision, since the fuselage obstructs 
extensive portions of the view where transparent windows would normally be located. 
Such a design would violate Chapter 14 of the Code of Federal Regulations (14 CFR) 

Part 25.773 which stipulates that “the flight deck windshield must provide sufficient 
external vision to the pilot to safely perform any maneuvers within the operating limits of 
the aircraft.” FAA Advisory Circular (AC) 25-773 further instantiates this requirement 
as stipulating a clear area of vision, as shown in Figure 3. 
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Figure 3: Clear Area of Vision as per AC25-773 


In addition, the FAA requires under 14 CFR Part 91.113, across all classes of airspace, 
that “when weather conditions permit, regardless of whether an operation is conducted 
under instrument flight rules or visual flight rules, vigilance shall be maintained by each 
person operating an aircraft so as to see and avoid other aircraft.” The lack of outside 
visibility due to the fuselage obstructing significant areas required to be “clear” would be 
further grounds to deny certification. 

Even if certification of this configuration were to be awarded, the lack of forward 
visibility by the pilot would - at best - severely restrict aircraft operations and airspace 
usage. For instance, it might be possible to receive special handling and stipulate that all 
flights and flight operations are flown under Instrument Flight Rules (IFR). The rationale 
would be that an aircraft without sufficient pilot visibility is essentially the same as one 
flying during Instrument Metrological Conditions (IMC). 

From an operational standpoint, the supersonic aircraft without an XVS or drooped noise 
would be incapable of flying under Visual Flight Rules (VFR) and would require “special 
handling.” Both of these conditions are operationally problematic, for example: 

• The lack of a VFR capability would prohibit operations to certain airspace 
classes and airports (e.g., see Drumm et al, 2004) where operational 
limitations around some airports and facilities cannot provide continual 
IFR services, such as those located in mountainous regions where radar 
coverage is not available. 

• The lack of a VFR capability can create inefficient flights during actual 
VFR conditions since pilots can no longer “cancel IFR” and go direct to 
their destinations or accept a visual approach clearance. IFR aircraft 
spacing and routing is dictated by Air Navigation Service Provider 
(ANSP) regulations and procedures. 
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• “Special handling” by the ANSP necessitates that the ANSP is integral to 
the planning and execution of all flights. The Concorde required “special 
handling” in the United States to meet certification requirements. While 
this was viable for the fourteen Concorde aircraft that operated world- 
wide, this scenario is not economically viable for any reasonable volume 
of supersonic aircraft. 

• Finally, emerging Next Generation Air Transportation System (NextGen) 
concepts run counter to the concept of “special handling.” These concepts 
are exploring ways to optimize traffic flow without direct ANSP control 
over flight operations and shifting some traditional ANSP roles and 
responsibilities for aircraft separation to the flight deck. 

Significant research was conducted under NASA’s High Speed Research (HSR) program 
during the 1990s on the design and development issues associated with an XVS for a 
conceptual HSCT aircraft. What emerged from this work - which still holds true today - 
is that the key challenge for an XVS design exists during VFR operations and when it is 
assumed that flight crew have natural visibility (even though they may be operating on an 
IFR flight plan). 

The driving XVS design standards emerge from the three tenets of VFR operations: 

1 . “see-and-avoid” 

2. “see-to-follow” 

3. “self-navigation” 

These VFR operational requirements apply to all aircraft, not just to low-boom 
supersonic aircraft. The issues and requirements for these other vehicles are discussed in 
the following sections as their experiences are applicable to the design of an XVS. 

As one example, an emerging challenge - being brought forth in the evolution of 
NextGen - is the concept of Equivalent Visual Operations (EVO). The EVO concept is 
notionally the ability to achieve or even improve on the safety of current-day VFR 
operations, maintain the operational tempos of VFR, and even perhaps, retain VFR 
procedures - all independent of the actual weather and visibility conditions. While EVO 
capability is not required for a successful supersonic aircraft, the technologies and 
operational requirements to develop EVO are in many ways equivalent to XVS. As such, 
current research in the use of synthetic and enhanced vision systems and other interface 
modalities as enabling technologies to meet the EVO operational concept are pertinent to 
this work. 

EVO technologies strive to create VFR operations for today’s aircraft when flying in 
actual IMC whereas XVS technologies try to enable an IMC cockpit (i.e., low-boom 
supersonic aircraft) to operate under VFR. 
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3. Design Standards Basis 

Design standards are drafted in the following on the basis that the XYS must provide 
equivalent levels-of-performance and safety to that of a pilot’s natural vision through a 
forward-facing window. 

First, equivalent levels of performance and safety emerge from “human vision 
equivalence.” Secondly, equivalent levels of performance and safety can be derived from 
the three tenets of VFR operations - “See-to-follow,” “See-and- Avoid,” and “Self- 
Navigation.” These VFR tenets are engrained in the FAA regulations and advisory 
materials. As such, these materials and associated analyses are used as a basis from 
which to define design standards for XVS display and sensor resolution. 

While three tenets to VFR operations must be met - based on past HSR and other 
research - the driving XVS requirements for sensor and display resolution are those 
necessary to enable “see-to-follow” and “see-and-avoid” operations. “Self-navigation” 
has not been found to be a driving factor and is not included in these standards 
development (e.g., see Summers, 1998). Self-navigation, in this context, includes the 
ability to identify and fly with respect to visual flight references (such as navigation with 
respect to cultural objects - roads, rivers, large man-made structures, etc) and the ability 
to conduct visual approaches, landings, and take-off operations. For example, flight 
testing has shown that pilots can safely conduct approach and landing operations, (i.e., 
“self-navigate”) even with severely restricted forward fields-of-view and minimal 
resolution (e.g., Perry, Dana, and Bacon (1967), Layton and Dana (1966), and Gaidsick, 
Dana, and McCracken (1969)) These works show that the tenet of “self-navigation” does 
not drive the resolution requirements of an XVS. 

The XVS design standards are derived using the following principles: 

1 . XYS design standards are first derived by requiring equivalent performance to 
that provided by the human visual system. These requirements are briefly 
reviewed in Section 4. 

2. Additional XVS design standards are established by assuming “equivalent” visual 
acuity from advisory material associated with “the pilot’s role in collision 
avoidance” - that is, “see-and-avoid.” Regulatory guidance is employed in 
defining XYS requirements for display and sensor resolution based on “see-and- 
avoid” in Section 5. 

3. Visual acuity requirements based on the capability to “see-to-follow” are 
presented in Section 6. “See-to-follow” is a capability - not a regulatory 
requirement. Therefore, advisory material does not exist, but one might consider 
that this requirement is derived from 14 CFR Part 25.1301 - satisfying the 
intended function of an XVS as being “functionality equivalent to that of a 
forward-facing window” and thus, enabling VFR operational capability. 

Background information for functional requirements for an XYS can also be found in 
Summers (2008). These functional requirements were derived during NASA’s HSR 
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program, using similar methodology. The HSR functional requirements are more 
comprehensive than those presented in the following; however, these HSR-XVS 
requirements are colored by operational assumptions invoked at that time and they are 
also predicated by the technologies and operational concepts prevailing and foreseen at 
that time (circa 1999). The requirements given in the following reflect the present 
regulatory and operational environment and should be considered an update to these 
HSR-XYS requirements. 
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4. Human Vision Equivalence 

Various factors influence the perceived image quality of an electro-optical sensor and 
display system including (Holst, 2008): 

• Sensor Performance 

Resolution 

Sensitivity 

- Noise 

Output/Input Transformation 

• Display / Monitor 

Resolution 

Luminance 

Contrast 

Distance from Observer 

• Scene Content 

Target Characteristics 
Background Characteristics 

- Lighting 
Motion 
Clutter 

• Atmospheric Transmittance 

Haze 

- Fog 
Rain 
Dust 
Clouds 
Etc. 

All of these factors are, of course, critical in the assessment and evaluation of an XVS 
image. However, only the effects of resolution for the sensor and display systems are 
considered directly in the following as they pertain to the ability of an XVS to provide 
equivalent performance and safety to that of forward-facing windows. 

Human vision “equivalent performance” is impossible to summarize in one or two 
concise requirements. Human vision performance is described in numerous textbooks 
(e.g., Westheimer, 1972); the details of which are not repeated here. Nevertheless, some 
basic standards associated with human visual performance are investigated as XVS 
design standards for display and sensor resolution. 

It should be noted that the effect of the aircraft windows are neglected in this analysis. 
While some optical distortion and vision modifications due to the transparencies are 
always present, these effects must be minimal during aircraft certification (see Chapter 14 
Federal Aviation Regulations Part 25.773) and are therefore, not considered a significant 
factor. In addition, testing during the HSR program showed that the influence of the 
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transparencies on human vision performance was in fact negligible (Quinn, Larson, 
Roush, and Johnson, 1996). 

The most commonly used, but not necessarily sufficient, measure of human visual acuity 
is provided by Snellen-type acuity tests. This standard is most often invoked as an XVS 
design standard. The rationale is simple: a) it is easily understood; and b) it is one for 
which virtually everyone has been tested; so familiarity is ensured. 

Display system performance is evaluated by viewing a Snellen acuity target (and others 
analogous to it, e.g., Landolt C) through the XVS sensor and display. Equivalent human 
visual acuity is said to occur if the user is able to demonstrate 20/20 vision (or 6/6 for 
metric assessments) - see Figure 4. 

By this measure, normal human visual acuity equates to the ability to resolve one arc 
minute. As a display image is created by pixels, it is often convenient to express visual 
acuity in terms of the number of pixels to define a degree (analogous to dots per inch 
(dpi) in printing). Thus, a display system resolution of less than 60 pixels per degree 
(ppd) would mean that the display is the “limiting” resolution - i.e., it cannot provide 
resolution at least equal to that of human vision. To relate Snellen acuity to display and 
sensor resolution, display resolutions which are “equivalent” to Snellen acuity values are 
listed in Table I. 



per cycle per cycle 


I 

60 pixels 60 arc min 

per degree per degree 

Figure 4: Snellen Acuity Related to Display Resolution 

To illustrate how this acuity test relates to XVS design standards, US broadcast television 
display standards are compared. (In these examples, the camera and display are assumed 
to be “loss-less” and they show their native resolutions without a magnification or 
minification - i.e., a so-called “unity” or “conformal” field-of-view - whereby the camera 
field-of-view subtends the same visual angle as the display when viewed from the 
observer eye point. The observer is assumed to sit 25 inches from the display.) 

In Figure 5, the effect of changing the field-of-view with a constant resolution 640x480 
video system is shown. The field-of-view is changed because a different monitor size is 
used as the display device while conformality is maintained. The abscissa shows the 
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diagonal dimension of the display. The data also shows the horizontal and vertical FOY 
displayed. (Note that this example does not use the 525 line resolution National 
Television Standards Committee (NTSC) video standard. Instead, Video Graphics Array 
(VGA) resolution of 640 horizontal (H) x 480 vertical (V) video is assumed to be 
equivalent for convenience.) 


Table I: Snellen Acuity Related to Display Resolution 


Snellen Acuity 

Min. Angle of 
Resolution (arc minute) 

Pixels per Degree 
Resolution 

20 / 200 

10 

6.0 

20/160 

8 

7.5 

20/125 

6.3 

9.5 

20 / 100 

5 

12.0 

20/80 

4 

15.0 

20/60 

3.2 

18.8 

20/50 

2.5 

24.0 

20/40 

2 

30.0 

20/30 

1.6 

37.5 

20/25 

1.25 

48.0 

20/20 

1 

60.0 

20/ 16 

0.8 

75.0 

20/12.5 

0.63 

95.2 

20/10 

0.5 

120.0 


The display resolution varies between 30 and 15 ppd for the 640x480 display system. 

This display system provides the Snellen-equivalent resolution of between 20/40 (12 inch 
diagonal) and 20/80 (25 inch diagonal monitor). The higher acuity is provided by the 
smaller display size because of the higher pixel density, providing only 20 degrees field- 
of-view. As the display surface increases in size, the same number of pixels (640x480) is 
now spread over a larger field-of-view. Thus, when this video is displayed across almost 
40 degrees field-of-view, the pixel density drops to 15 ppd; hence, the poorer visual 
acuity. 


11 


NTSC (4:3) Resolution 
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Figure 5: Visual Acuity and FOV vs. Display Size, Using NTSC Resolution 

High Definition Television (HDTV) video examples are shown in Figure 6. In these 
examples, HDTV is portrayed using 1920 H by 1080 V pixel resolution. The increased 
display resolution from HDTV provides between 8 1 and 40 ppd for monitors sized 
between 12 inch and 25 inch diagonal. These displays provide a Snellen-equivalent 
acuity better than 20/15 (15 inch diagonal monitor) and better than 20/30 (25 inch 
monitor). This does not imply that a person with 20/20 vision will be able to have 20/15 
vision by using this system. Rather, the system has enough resolution to accurately 
display a scene with 20/15 acuity. This resolution improvement over VGA clearly 
highlights the attraction for the HDTV revolution in the US. The data shows that 20/20 
Snellen acuity is provided by a “loss-less” HDTV system if the display is 16 inch 
diagonal (or smaller), subtending ~20° V x 30° H FOV. 
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HDTV (16:9) Resolution 



Figure 6: Visual Acuity and FOV vs. Display Size, using HDTV Resolutions 

Snellen acuity must be considered a necessary, but perhaps not sufficient, prerequisite for 
the XVS display and sensor design standard. As mentioned before, Snellen acuity is just 
one measure of human vision performance. For instance, the test employs “full” contrast 
targets; that is, the test uses a fully lit black “target” with a white background. The 
Snellen acuity represents the spatial frequency at which the observer’s eye can no longer 
discriminate differences in the light and dark transitions in the image. Human visual 
acuity and discrimination associated with color and variations of contrast are not 
considered at this time. 


XVS acuity should optimally be evaluated using Modulation Transfer Function (MTF) 
testing. Display contrast (C) is defined as: 


C 


_ ( An;i\ An in ) 

(Anax + Anin ) 


where L max is the maximum luminance and L m i n is the minimum luminance of the 
displayed imagery. The MTF measures the displayed image contrast compared to the 
input image contrast as a function of spatial frequency (Holst, 2008). Because the eye is 
normally considered to require an image contrast of approximately 3% in order to 
distinguish the light-to-dark transitions in the bars, the limiting resolution using a 
Snellen-type acuity test can be considered as the spatial frequency at which the MTF has 
fallen to a value of three percent. Thus, the Snellen acuity test represents just one point on 
a MTF. The MTF would capture the contrast sensitivity performance of the XVS display 
and sensor system. 
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Ideally, the MTF should be used as a basis for XYS display system performance 
evaluation. At this time, however, MTF criteria for XVS applications have not been 
established. 

Even if 20/20 Snellen acuity is measured, other human vision capabilities suggest much 
higher pixel densities than the 60 ppd Snellen-equivalent are necessary (Hooper, 2000). 
Glint, vernier line comparisons, etc. suggest pixel densities as high at 1500 ppd might be 
required. Thus, XYS resolution and system performance to achieve 20/20 Snellen acuity 
or better (60 ppd) must be considered a minimum but perhaps not sufficient XVS 
prerequisite. Other test parameters should also be considered (e.g., see Sweet and 
Giovannetti, 2008 and Winterbottom et al, 2004). 

Even if Snellen-equivalent performance is demonstrated, it may be difficult to prove 
human vision equivalence, especially since some human vision performance capabilities 
far exceed that required for Snellen vision equivalence (Hooper, 2000). Functional 
performance requirements, such as those in Sections 5 and 6, may be necessary to 
demonstrate equivalent performance. 
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5. Visual Acuity for “See-and-Avoid” 


The flight rules prescribed in Part 91 of the Federal Aviation Regulations (FARs) set 
forth the concept of “see-and-avoid.” See-and-avoid serves three primary functions 
(Australian Transport Safety Board, 1991): 

1 . Self-separation of aircraft outside controlled airspace 

2. As a separation procedure for YFR aircraft in control zones, where the pilot is 
instructed to see and avoid another aircraft. This procedure only operates when 
the pilot can see the traffic and is therefore significantly different to other types of 
see-and-avoid which may involve unalerted searches for traffic. 

3. Last resort separation if other methods fail to prevent a conflict, regardless of the 
nature of the airspace. 

The merit of see-and-avoid operations and the associated human visual, human attention, 
and operational issues have been studied extensively (e.g., see Australian Transport 
Safety Board, 1991; Graham, 1989). Whether see-and-avoid actually is effective or not is 
immaterial. The FAA requires under 14 CFR Part 91.113 across all classes of airspace 
that “when weather conditions permit, regardless of whether an operation is conducted 
under instrument flight rules or visual flight rules, vigilance shall be maintained by each 
person operating an aircraft so as to see and avoid other aircraft.” 

For certification of an XVS, a formal safety analysis would be performed to quantify the 
required equivalent levels of safety for see-and-avoid. This type of safety analysis to the 
see-and-avoid problem is currently being performed by the Uninhabited Aerial Vehicle 
(UAV) community (e.g., see Kuchar, 2004). Their work is of great interest to an XVS 
design problem, but to apply this analysis directly to an XVS design, is difficult at best. 
The difference is that the UAY community is trying to develop “sense-and-avoid” 
whereby “see-and-avoid” is provided by a combination of sensors and computer 
processing, not a human pilot. Thus, the difficulty arises primarily from trying to 
sufficiently quantify and validate XVS performance as it impacts the “visual acquisition” 
component of this analysis. In sense-and-avoid, the process is deterministic (i.e., the 
sensor either can or cannot acquire and track other traffic to avoid). In see-and-avoid, the 
unquantifiable capabilities (and limitations) of the human are involved. 

The visual acquisition task in see-and-avoid is, in essence, a “target detection” problem 
of which a wealth of military data and research are available. Target detection research 
shows that the probability that a target will be recognized (by a human observer) is a 
function of: 

• the probability that the observer is searching an area that is known to contain a 
target, looks with his/her foveal vision for a specified glimpse time (1/3 s) in the 
direction of the target; and, 

• the probability that the displayed target image is viewed foveally for one glimpse 
period with sufficient contrast and size to be detected; 
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In the development of these XYS design standards, the issues associated with how and 
how efficiently a pilot “visually scans” a display is ignored even though it is an important 
design issue (e.g., see Boff and Lincoln, 1998). For instance, the amount of time to 
visually scan a larger field-of-regard to locate a target is theoretically longer than 
scanning a smaller field-of-regard. 

Also, these analyses do not consider the role of surveillance alerting that can alter and/or 
improve visual search importance, vigilance, and localization (e.g., where surveillance 
alerting might be generated from sources such as Air Traffic Control surveillance radar or 
Traffic Collision Alerting Systems). Data suggest that an alerted search is eight to nine 
times more successful in visual acquisition than an un-alerted search (Andrews, 1991 and 
Boff and Lincoln, 1988). 

The analyzes shown here only consider if a visually displayed target image is of 
sufficient contrast and size for visual detection. This component of human visual target 
acquisition is shown to be directly proportional to the target angular size and the apparent 
target-background luminance ratio (Boff and Lincoln, 1988). Thus, the resolved target 
contrast and size are the only determinants used for establishing the required XVS 
display and sensor resolution. 

A series of experiments (Ratches et al, 1997) attempted to quantify the target detection 
probability as a function of the resolution of the sensor/display system (i.e., the target 
angular size). This work showed that, whether using an unaided eye or other sensors, 
such as night-vision goggles or thermal imaging systems, four tasks were involved in the 
target acquisition process: 

1) Detection: correctly discriminating an object in the image from background 
and system noise. 

2) Orientation: correctly determining the detected target aspect or direction of 
movement. 

3) Recognition: correctly determining the class membership of the target. For 
example, is it a truck? A tank? etc. 

4) Identification: correctly determining the exact identity of the target, e.g., for 
automobiles, is it a Ford or Chevrolet? 

These same and similar processes in detection, recognition, and identification for air-to- 
air target acquisition tasks have also been found (e.g., see Rohrer, 1996). 

Experimental data led to the development of Johnson’s criteria for target acquisition 
(Ratches et al, 1997). Johnson’s work presumed that the ability of the observer to detect, 
recognize and identify a target were a function of how well a critical dimension of the 
target could be resolved by the sensor-display system. An example of the data is shown in 
Figure 7, indicating the probability of detection and recognition (of a man or a vehicle) is 
a function of the number of sensor-display cycles (sensor/display line pairs) depicting the 
object. 
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Figure 7: Probability of Detection and Recognition vs. System Resolution 

(from Ratches, 1997). 

Johnson’s work evolved into straight-forward criteria for display and sensor requirements 
for visual acquisition as shown in Table II. The “critical target dimension” was based on 
intuition and was usually chosen to be the minimum dimension. 


Table II: Johnson's Criteria - Resolution Requirements 


Task 

Line Resolution per Minimum 
Dimension 

Detection 

1.0 ± 0.25 line pairs 

Orientation 

1.4 ± 0.35 line pairs 

Recognition 

4.0 ± 0.8 line pairs 

Identification 

6.4 ± 1.5 line pairs 


These criteria only pertain to the image size of the visual target detection task; that is, the 
target angular size and the associated resolution of the display and sensor to generate the 
image for human visual acquisition. The other component of the visual acquisition task - 
the apparent target-background contrast - is dependent upon many other factors such as 
atmospheric conditions (clouds, fog, haze, moisture content, etc.), target color/material, 
background illumination and color, sun angle, background (sky/ground), etc, and 
sensor/display contrast performance. Even though these are critical issues to the task, 
these are not influenced by the display resolution directly and therefore, are not 
considered herein. 

Johnson’s criteria has been shown to be a rough, first-order approximation as numerous 
analyses have shown it to be flawed in several respects (McDonald and Vorst, 2002). 
Other, more elaborate analyses may more accurately predict human detection 
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performance and should be referenced (e.g., see Aviram and Rotman, 2000 and 2001, 
Vollmerhausen, Jacobs, and Driggers, 2004); however, for the comparative analyses in 
the following, Johnson’s criteria should be sufficient as it provides an intuitive measure 
for display and sensor resolution effects on visual target acquisition. 

5.1 Resolution Requirements Based on See-and-Avoid and Johnson’s Criteria 

Johnson’s criteria are used with FAA advisory circular material to derive XYS resolution 
requirements to meet see-and-avoid precepts. The objective is to provide a rough order 
of magnitude assessment of XVS resolution requirements for equivalent safety and 
performance levels in the absence of a formal safety analysis. 

Under FAA AC90-48C - Pilot’s Role in Collision Avoidance - pilot recognition and 
reaction times are published for detection, decision, and evasive action in a see-and- 
avoid, mid-air collision scenario. These times are shown in Table III. 

Table III: Recognition and Reaction Times - AC90-48C 


Recognition and Reaction Tasks 

Times 

See Object 

0.1 sec 

Recognize Aircraft 

1.0 sec 

Be Aware of Collision Course 

5.0 sec 

Decision to Turn Left or Right 

4.0 sec 

Muscular Reaction 

0.4 sec 

Aircraft Lag Time 

2.0 sec 

Total: 

12.5 sec 


The recognition and reaction times stipulated in Table III are applied along with 
Johnson’s criteria (Table II) to develop see-and-avoid XYS display resolution 
requirements using the following guidelines and assumptions: 

1 . From Table II, the number of pixels required for detection, orientation, 
recognition, and identification are equal to 3, 4, 9, and 14, respectively (i.e., 
approximately twice the number of line pairs plus one). 

2. From Table III, a minimum of 12.5 seconds is needed between when detection of 
an aircraft is made and a mid-air collision can be averted. This function 
corresponds to the detection task in accordance with Johnson’s criteria (Table II). 
This time defines the distance between two aircraft on a collision course which, of 
course, depends on their closure speeds and geometry. The XVS resolution is 
determined by this distance and the fact that 3 pixels of the XVS sensor/display 
system must resolve the target aircraft. 

3. From Table III, the pilot has a minimum of 6.4 seconds before the collision to 
recognize that the two aircraft are on a collision course and maneuvering is 
required to avert the collision. This function corresponds to the orientation task in 
accordance with Johnson’s criteria (Table II). Depending upon the closure rate 
between the aircraft, this time defines the distance between the two aircraft. The 
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XYS resolution is determined by this distance and the fact that 4 pixels of the 
XVS sensor/display system must resolve the target aircraft. 

A worst case, head-on mid-air collision scenario is assumed (Figure 8). 





Figure 8: God’s Eye View of Worst-Case Head-On See-and-Avoid Scenario 


The most demanding mid-air scenarios requiring see-and-avoid occur in uncontrolled 
airspace below 10,000 ft Mean Sea Level (MSL) (e.g., see Drumm et al, 2004). 
Assuming worst-case, ownship and the “target aircraft” are flying at 250 knots indicated 
airspeed (KIAS) in opposite directions at 10,000 ft altitude generating a closure rate of 
578 knots. 

Three general classes of aircraft “size” were arbitrarily assumed for this analysis. These 
are shown in Figure 9. These aircraft were picked to demonstrate the influence of target 
size on the “see-and-avoid” problem. They scale by approximately a factor of 2. Note 
that the critical dimension under Johnson’s criteria is the minimum distance (aircraft 
height for the head-on collision scenario), but in this work, both vertical and horizontal 
(wing span) extent of the target aircraft are used. Other analyses use the target cross- 
sectional area (e.g., Andrews, 1991). 

Under these assumptions, the XVS resolution requirements are shown in Table IV. 
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‘Big” (B-757-200) 



“ Medium ” (ERJ-145) 



“Small” (C-172) 



Span 


Aircraft 

“Class” 

Span 

Height 

Length 

“Big” 

124.8 ft 

44.5 ft 

178.6 ft 

“Medium” 

65.8 ft 

22.2 ft 

98.0 ft 

“Small” 

36.1 ft 

8.9 ft 

27.2 ft 


Figure 9: Dimensions of Aircraft in See-and-Avoid Analysis 


With a head-on closure in this analysis, detection of the target aircraft has to occur 
slightly inside of 2 nmi separation to meet the guidance of AC90-48C. Recognition has 
to occur slightly inside of 1 nmi separation. These separation distances are not 
unreasonable. They generally meet the expectations of subject matter experts (i.e., 
experienced intercept pilots, Rohrer, 1996). 


Table IV: See-and-Avoid-Based XVS Resolution Requirements 



Detection 

Orientation 

Vehicle Range 

12210ft 

6252ft 

Size Resolution Reqr ’d 

3 pixels 

4 pixels 

Big Height 

14.4 ppd 

9.8 ppd 

Span 

5.1 ppd 

3.5 ppd 

Medium Height 

28.8 ppd 

20.0 ppd 

Span 

9.7 ppd 

6.7 ppd 

Small Height 

71.8 ppd 

50.0 ppd 

Span 

17.7 ppd 

12.1 ppd 


The XVS resolution requirements, based on Johnson’s criteria, show that the detection 
task is the most demanding. 

The XVS resolution requirements also show that the required resolution in pixels per 
degree (ppd) is highest for the smallest vehicle. This result is intuitively obvious. The 
data show that a C- 172-sized aircraft on a head-on collision course would require more 
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than 70 ppd resolution for the pilot to correctly detect the target on the XYS in time to 
avert a mid-air. This result would initially seem peculiar based on a C-172 flying at 250 
KIAS. However, Very Light Jets, such as the Eclipse, are not radically different in size 
from a C-172 (or military jets for that matter). So the results are relevant. 

5.2 Adequacy of See-and-Avoid Based on FAA AC90-48 

An alternative analysis is performed to assess the FAA’s published timeline for evasive 
action in the see-and-avoid/collision avoidance task. 

Under emerging standards for “sense-and-avoid” systems (ASTM, 2007), sensing must 
be sufficient that “detection of the collision threat shall be at a range to allow a resolution 
maneuver that results in a required miss distance of 500 ft or greater.” 

In this analysis, the same “worst-case” head-on collision scenario is assumed but a 500 ft 
collision resolution requirement is imposed. XVS resolution requirements are again 
derived and the implications of this requirement can be judged against FAA AC90-48C. 

Two aircraft - ownship and an intruder - are flying in opposite directions at co-altitudes. 
Both ownship and the target aircraft are flying at 250 KIAS at 10K ft (578 knot closure 
rate). The vehicle sizes are varied as per Figure 9. Ownship maneuvers to avoid the mid- 
air collision but the target aircraft does not. 

The analysis is conducted in reverse sequence for simplicity. The 500 ft separation 
distance is required and is used as the final time. The preceding times (and distances) for 
ownship to achieve this separation standard are calculated as shown in the following 
figure. The final condition is sketched in Figure 10. 



Figure 10: God's Eye View of Final Encounter Position 

To achieve this final position, constant altitude turns using 3 values of normal 
acceleration for ownship are used as a separation maneuver. The normal acceleration 
values are 1.25g, 1.50g, and 2.0g - not atypical for transport category aircraft. For 
constant altitude, constant speed conditions, the turn rate and turn radius are computed 
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from kinematics. Ownship is flying at 250 KIAS which equates to 488 ft per second true 
airspeed (v). The bank angle ($) required to achieve the target normal acceleration values 
are: 


2 levels 

Bank Angle ( dA 

1.25 g 

37 deg 

1.50 g 

48 deg 

2.00 g 

60 deg 

... , Stan (d>) 

turn rates ( <p ), equal to — — 


V 

2 levels 

Turn Rate ( & ) 

1.25 g 

2.8 deg/sec 

1.50 g 

4.2 deg/sec 

2.00 g 

6.5 deg/sec 


v 2 t . 

(R), equal to 

. ,,x’ beln § : 


g tan(^) 

g levels 

Turn Radius (R) 

1.25 g 

9850 ft 

1.50 g 

6613 ft 

2.00 g 

4270 ft 


are as follows: 


The final geometry for the separation between ownship and the non-maneuvering target 
is sketched in Figure 1 1. A 500 ft radius circle is assumed around the target at the final 
position. Ownship has been turning at a constant altitude, constant speed, and constant 
turn rate. The turn rate and radius of turn, R, depended upon the g-level of the 
separation, as shown in the data above. 
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Figure 11: God's Eye View of Encounter Geometry 


For this final position, the two circles are tangential to each other. Two circles with 
centers at (x, ,y, ) and (x 2 , y 2 )with radii of /y and r 2 , are mutually tangent if 

(xj - x 2 ) 2 + (>’, - y 2 ) 2 = (p + r 2 ) 2 (Weisstein, 2009). For the target aircraft, the 
separation circle is xj = yj =0 with /y = 500 ft. Knowing that >y = R, the position X 2 is 

derived as x 2 = -^/(500 + i?) 2 - R 2 . 

The heading angle (p) that ownship must turn to create this separation geometry is 
computed as arctanfa/R). The time to compute this turn is computed using the turn rate 
for each g-loading, as follows: 


g levels 

Turn Angle (n) 

Time to Turn r| 

1.25 g 

17.9 deg 

6.3 sec 

1.50 g 

21.6 deg 

5.1 sec 

2.00 g 

26.5 deg 

4.0 sec 


To simplify the analysis, it was assumed that 3 seconds prior to starting this separation 
maneuver, the ownship aircraft begins rolling to the target bank angle (either 37, 48, or 
60 degrees) but without generating any turn rate or lateral separation. This assumption is 
pessimistic and admittedly not realistic; however, it makes the calculations trivial. 

The time from the beginning the rolling maneuver to achieve a 500 ft separation and the 
distance between ownship and a non-maneuvering target aircraft is given in Table V. 

Table V: Time and Distance to Achieve 500 ft Separation in Head-On Scenario 


g-Level 


Time to Complete 


Distance Between 
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Aircraft 


Separation 

Maneuver 


1.25 

9.3 sec 

9237 ft (1.5 nm) 

1.5 

8.1 sec 

8096 ft (1.3 nm) 

2.0 

7.0 sec 

7061 ft (1.2 nm) 


This simple analysis shows that between 9.3 and 7.0 seconds are required to achieve 500 
ft separation in a worst-case, mid-collision scenario. The data of Table V also show that 
the higher g-levels create the required separation in shorter times - as one would logically 
expect. 

More importantly, these times are significantly longer than the 2.4 seconds required in 
AC90-48C (Table III) to begin the separation maneuver. At best, 4.6 seconds more time 
is needed to create the 500 ft separation than stipulated in AC90-48C. This extra 4.6 
seconds - for a total of 17.1 seconds for the see-and-avoid process - equates to 4494 ft 
additional separation between the two aircraft required in the detection and orientation 
phase. These new see-and-avoid distances suggest that that the XYS resolution would 
have to be closer to 98 ppd or 84 ppd for successful detection and orientation, 
respectively, of the “small” aircraft’s most critical dimension (instead of the 71.8 ppd and 
50.0 ppd shown in Table IV). 

Formal safety analysis, based on various sources to mathematically model and predict 
human and sensor performance (i.e., determine a probability of detection) in “see-and- 
avoid” environments, should be performed. These types of analyses (for instance, a Mid- 
Air Collision Assessment Tool (MARC AT®)) have generated probabilistic risk 
assessment, providing objective data on collision scenarios and risk mitigating avoidance 
strategies (see Schaefer, 2004). 

The analyses of Schaefer (2004) suggest that the human vision process for see-and-avoid 
is inadequate to prevent mid-air collisions. This work and others (Drumm et al, 2004) 
suggest that 12.5 seconds may not be sufficient for detection and maneuvering to create a 
500 ft separation. The analyses performed herein and the data of Table V corroborate 
these modeling efforts. 

While the inadequacies of human vision (and vigilance) for see-and-avoid are well- 
documented (Australian Transport Safety Board, 1991; Graham, 1989), it should be 
noted, however, that the analytical models of Schaefer (2004) are not supported by 
accident data statistics. The accident statistics are not as bad as implied by the analytical 
tool results. 

In summary, the see-and-avoid analyses show that: 

• See-and-avoid is problematic - human observers are neither very efficient nor 
continually vigilant - but the accident data, especially with the advent of 
automatic surveillance and alerting services, such as Traffic alert and Collision 
Avoidance System, and increased pilot training and awareness does not indicate 
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that it is problematic. In any event, see-and-avoid is a regulatory requirement as 
per 14CFR Part 91.113. 

• Target detection primarily depends upon the displayed target size and contrast 
compared to its background and the atmospheric conditions through which the 
target is observed. A simple, rough order of magnitude approximation to quantify 
an observer’s ability to detect a target, determine its orientation and direction of 
flight, recognize its type, and identify the target is provided by Johnson’s criteria 
(Table II). 

• AC90-48C offers times required for target detection and determining its collision 
course in time to avert a mid-air collision (Table III). A worst-case, head-on mid- 
air course was assumed and, using Johnson’s criteria, shows that 72 ppd for XVS 
camera and display resolution are required. 

• Analyses for a head-on collision also shows that the times shown in AC90-48C 
are optimistic. To create a 500 ft separation, a target should be detected almost 17 
seconds - not 12.5 seconds - before the collision point. The XVS resolution to 
detect the target at this range is almost 98 ppd. This requirement, however, is 
unsubstantiated by present accident data statistics or certification advisory 
material. 
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6. Visual Acuity for “See-to-Follow” 


“See-and-avoid” is a regulatory requirement applicable to all aircraft, in all airspace 
classes. On the contrary, “see-to-follow” is an operational capability - one of the tenets 
of YFR operations. It might be possible to operate an XVS-equipped aircraft strictly on 
an IFR basis; however, its economic viability could be marginal. For this reason, the 
display/sensor resolution requirements for “see-to-follow” operations are briefly 
examined in the following. 

The applicability of Cockpit Display of Traffic Information (CDTI) for aiding pilot 
separation in this task is not analyzed in the following analyses. Obviously, these 
electronic aides (e.g., Prinzo and Hendrix, 2003, McAnulty and Zingale, 2004, and Bone 
et al, 2004) would significantly ease the task, at a minimum, and might eliminate the need 
for natural vision in the future. But since these new operations are not yet operational nor 
can it always be assured that these types of non- visual separation procedures would be 
used, the “worst-case” scenario of using actual vision (albeit provided by an XVS 
display) will be analyzed. 

Minimum separation and spacing requirements for today’s “see-to-follow” operations can 
be derived from current wake turbulence standards and procedures. As per US FARs, a 
pilot, when sshe has accepted ANSP instructions to visually follow another aircraft, is 
responsible to maintain in-trail separation and to accept responsibility for wake 
turbulence separation. Standard separation from other than a “heavy/B757” aircraft is 3 
miles. Wake turbulence separation of a small aircraft from a “heavy/B757” is 5 miles. 

Johnson’s criteria is again used as a rough, first-order approximation to the visual task 
requirements for “see-to-follow.” Unlike the visual target acquisition task, the corollary 
between Johnson’s criteria using detection, orientation, recognition, and identification 
phases (Table II) to the see-to-follow task is not obvious. 

In this analysis, the following assumptions are used: 

• The search is likely a directed search; that is, the flight crew knows the general 
direction in which to look for the traffic. This directed search is most often 
obtained from direct ANSP radio communication (e.g., “follow traffic on base, 3 
miles at your 2 o’clock, level at two thousand”). Since directed searches are up to 
9 times more effective than un-alerted searches (Andrews, 1991 and Boff and 
Lincoln, 1988), the detection task is not included in this analysis. (It is assumed 
to be successful.) In addition, this task may occur at distances much greater than 
the 3 to 5 mile separations. 

• The flight crew, focused on a detected object, must correctly determine that the 
object is an aircraft and it must correctly determine its trajectory. This task 
should correspond to Johnson’s “recognition” task. 

• Following recognition, the flight crew should positively identify that the aircraft it 
is following is indeed the correct aircraft. This task should correspond to 


26 


Johnson’s “identification” task, whereby the flight crew determines that the 
aircraft is a C-172, and not a Beech King Air. 


Under these assumptions, the XVS resolution requirements are derived by using the 
worst-case requirements - the smallest aircraft at the greatest distances. These are 
approximated by using the “small” C-172 dimensional data at a 3 mile separation 
standard and the “big” B-757 dimensional data at a 5 mile separation standard (see Figure 
9). The results of these analyses are shown in Table VI. 


Table VI: See-to-Follow-Based XVS Resolution Requirements 



Recognition 

Identification 

Vehicle Range 

15840ft 

15840ft 

Resolution Reqr ’d 

9 pixels 

14 pixels 

C-172 Height (8.9 ft) 

300 ppd 

467 ppd 

Length (27.1 ft) 

90 ppd 

140 ppd 

Span (36.1 ft) 

69 ppd 

108 ppd 



Recognition 

Identification 

Vehicle Range 

26400ft 

26400ft 

Resolution Reqr ’d 

9 pixels 

14 pixels 

B-757 Height (44.5 ft) 

90 ppd 

140 ppd 

Length (178.6 ft) 

23 ppd 

36 ppd 

Span (124.8 ft) 

33 ppd 

52 ppd 


The minimum dimension for both a B-757 and C-172 is its height. Using these numbers, 
the XVS resolution requirements to recognize and identify a C-172 at 3 miles requires 
300 to 467 ppd, respectively. These resolution requirements are 5 and 8 times the 60 ppd 
equivalent of Snellen visual acuity. Pilot opinion suggests that a C-172 cannot be seen 
head-on at 3 miles. Consequently, these resolution requirements don’t seem to 
reasonably approximate human vision equivalent. They far surpass them and hence, 
would unreasonably tax the design requirements for an XVS. 

More likely, the recognition and identification requirements for see-to-follow depend on 
the perception of the length and span of the aircraft. A target aircraft, once off of the pure 
head-on/tail-on aspect, begins to show significant visual area (i.e., its wing and body). 
Using the smaller dimension of the length or span, the worst-case XVS display resolution 
requirement requires between 69 to 140 ppd to recognize and identify a C-172 at 3 miles. 
The B-757 at 5 miles requires less display resolution since its much larger visual area 
more than offsets the additional 2 miles of range between it and the ownship vehicle. 
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7. Technology Trends and Associated Research Needs 

The preceding analyses show that: 

a) 60 ppd may meet a Snellen visual acuity equivalence but other human visual 
properties require higher resolution. 60 ppd and Snellen equivalence to 20:20 
vision should be an absolute minimum requirement. 

b) Assuming see-and-avoid scenarios and applying associated AC material, higher 
resolutions, nearing 100 ppd, are required; and, 

c) See-to-follow capabilities require even higher display/sensor resolutions 
because greater distances are involved. A rough, first-order approximation shows 
that up to 140 ppd is required to meet see-to-follow operations. 

Under the HSR program, designs to achieve 60 ppd XYS display were investigated. 
Methods to achieve these resolutions were extremely difficult, given the hardware 
available at that time. 

HDTV equipment was in its infancy during the time period of the HSR program. 
Prototype HDTV equipment was procured and flight-tested. As shown in Figure 12, a 
Cathode Ray Tube HDTV monitor, weighting approximately 175 lbs, was installed in the 
Total In-Flight Simulator (TIFS) flight test vehicle. The monitor was 22.75" wide x 
12.75" high, providing approximately 49° H x 29° V field-of-view from a 25" design eye 
reference point (DERP) view. This monitor provided approximately 40 ppd. With this 
resolution, a Snellen acuity of approximately 20/30 should be yielded. Field-tests of 
Snellen acuity showed between 20/40 and 20/50 performance. The differences between 
theoretical and actual acuity is likely due to imperfection in the video performance of the 
HDTV monitor and camera systems - i.e., the system was not “loss-less.” MTF tests 
were not performed. (In addition to the forward field-of-view HDTV monitor, an in- 
board FOV monitor was also installed to meet foreseen FOV requirements - see 
Summers, 1998.) During this flight test, see-to-follow or see-and-avoid tasks were not 
performed. The evaluations primarily emphasized self-navigation tasks. Self-navigation 
with this Snellen acuity was not a problem. 
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Figure 12: TIFS Flight Test HDTV XVS Display 

Following this flight test, a prototype XVS was flight-tested. This prototype hardware is 
shown in Figure 13. The desired 60 ppd resolution display was met, albeit only over a 
small field-of-view. The overall size of the display met HSR field-of-view requirements. 
The monitor provided 40° H x 50° V field-of-view from a 25" DERP, using three (3) 
tiled, 1280x1024 projectors with a rear-screen projection. The 60 ppd inset was only 21° 
H x 17° V - smaller than today’s typical Head-Up Display field-of-view. The display 
resolution outside of this inset was only about 33 ppd. Symbology was overlaid on the 
XVS video by use of chroma-keying. 



Figure 13: FL5 Flight Test XVS Prototype Display 

During this flight test (the so-called “FL5” flight test) a fairly elaborate series of see-to- 
follow and see-and-avoid flight test maneuvers were flown. Unfortunately, significant 
portions of the data, especially the human performance aspects of this test were not 
analyzed because the HSR program ended just at the conclusion of the FL5 flight test. 
However, this work did feed Image Object Detection (IOD) algorithm development as 
part of the “see-and-avoid” functionality for HSR (see Gandhi et al (2003) and Yang 
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Rift 




(2000)) which continues today as essentially part of the UAV “sense-and-avoid” 
program. 

This flight test also provided valuable development and validation of flight test 
maneuvers to operationally characterize and assess the performance of an XVS (or other 
vision-based system devices) for see-and-avoid and see-to-follow operations. In 
Appendix A, the most pertinent of these tests maneuvers are described. 

Since the NASA HSR program, display technology has significantly progressed in areas 
which impact resolution. These advances have primarily been driven from four thrusts. 

1 . HDTV has emerged as a commercial standard. As such, HDTV resolutions in 
many monitor sizes are now available at very affordable pricing. 

2. Video refresh rates for these HDTV systems are often designed to meet the 
demanding video-gaming / entertainment user, not just the 24 Hz cinematic 
requirement. 

3. The medical industry is moving toward eliminating film and using digital record 
archival. This change necessitates that digital storage and display resolutions 
approach that of X-Ray films. 

4. The cost of Field Programmable Gate Arrays and Digital Signal Processors has 
dropped by orders of magnitudes and programming of these devices is now a 
relatively easy and inexpensive task. Custom video hardware and software 
applications are now an “everyday” event so affordable merging, tiling, and 
seamless integration of video streams is no longer a technological hurdle. It’s just 
an engineering task. 

The state-of-the-art in today’s technology is so-called “Quad-HD” - that is, four times 
HDTV resolution or 3840x2400 pixels. Numerous venders are releasing commercially 
available Quad-HD monitors in various sizes, but so far, none have been in the XVS 
required form-factor. 

In 2001, a limited run of Quad-HD resolution monitors - the IBM T221 (Figure 14) — 
was made. The flat panel displays provide over 90 ppd resolution across an approximate 
40° H x 26° V FOV, when viewed from 25 inches DERP. 
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Figure 14: IBM T221 Quad-HD Resolution Display 

The T221 monitor came in different versions and was sold by various vendors. The 
versions differed by their installed video cards and digital video interfaces which 
influenced the resultant resolution and associated refresh rates. The native refresh rate 
was reported to be 48 Hz. The refresh rates could vary from 13 Hz up to 48 Hz. An 
Iiyama AQU561 ID liquid crystal display, which is based on the IBM T221 Model 9503- 
DG3 monitor, was used as the display for an XVS proof-of-concept evaluation. 

Under Department of Defense funding, two cameras were specially built to drive the IBM 
T221 monitor directly, exercising their native resolution. This Quad-HD camera (Figure 
15) utilizes custom field programmable gate array circuitry to generate and drive the IBM 
display with real-time video. Unfortunately, MTF or other similar test data have not been 
found documenting the results of using this camera and display system. 

If the native resolution of the display and a “loss-less” video system is assumed, better 
than 20/15 Snellen acuity could be provided by this system. This monitor and display 
system would nearly meet the projected XVS requirements for see-and-avoid and see-to- 
follow. 

One limitation with this camera and display combination for an XVS application is the 
inability to insert head-up symbology onto the video. The camera directly drives the 
display so chroma-keying (i.e., symbology insertion) cannot be easily performed. The 
camera would have to be mounted behind a collimated display (e.g., a Head-Up Display) 
to create overlaid symbology. 
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Figure 15: Quad-HD Camera 


7.1 XVS Proof-of-Concept Evaluation 

An alternate means of creating an XVS display using the T221 Quad-HD monitor was 
developed. The goal was to quickly assess the problem of adding symbology while 
achieving display resolutions near those required for an XVS. 

Four separate HDTV video cameras (Sony Model EV1-HD1) were used to drive the four 
HDTV inputs to the Quad-HD monitor. (For laboratory testing purposes at this time, the 
problem of aligning these four separation cameras to make a seamless single image is not 
important to this analysis.) 

To add symbology to this monitor, four HDTV frame grabbers (Foresight Accustream 
170) were installed into a PC. The sketch of this system is shown in Figure 16 




IBM QuadHD T221 Monitor 
(Driving Entire Display) 


Figure 16: Quad-HD XVS Proof-of-Concept System 

Snellen acuity tests were conducted with this system. Snellen acuity of 20/20 was 
obtained. These data are slightly worse than expectations but not terribly so. Rigorous 
analysis of the resolution results, such as the computations of the MTF, has not yet been 
conducted. The cameras used in this set-up are not state-of-the-art by any means. They 
were selected to provide a convenient 1920x1080 video source. 
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While the primary objective of this test was an XYS resolution proof-of-concept, a 
critical constraint to this design is the system latency. A Snellen acuity test is a static 
image test, however, the content of the XVS display is dynamic. Camera systems have 
latencies associated with acquiring an image and sending the video signal of the scene. In 
addition, the monitor has refresh updates and video bandwidth constraints. Displaying 
symbology on the XVS image can also introduce another source of lag or latency as the 
symbology is computed and merged with a frame grabber image. The combined effect of 
all these time delays can have a significant effect on visual perception and pilot 
performance during use of the XVS. The latencies must be considered in determining the 
overall effectiveness of the XVS system. If latencies are too high, the system can be un- 
flyable. 

An end-to-end latency test (camera acquisition, transmission, capture, process via frame 
grabber, symbology overlay and display interface and refresh) was conducted. Timer 
software was written and displayed on the Quad-HD monitor. This timer is then viewed 
by the camera system and displayed on the same computer monitor via frame grabber 
software. The camera image is captured by a frame grabber card. The total camera system 
latency can be determined by subtracting the computer displayed time minus the frame 
capture time. 

In Figure 17, a screen capture of the computer timer and the same timer captured and 
rendered by the camera and display system are shown. The T221 monitor was configured 
for a display resolution of 3840x2400 at 13Hz. Each camera was outputting a 1080i, 

60Hz format signal that was captured by a frame grabber. The camera latency was 
measured as the difference between the two time displays. The system latency was 
between 300 and 330 milliseconds. This latency has been found to be unacceptable 
(Bailey, Arthur, and Williams, 2004). A reasonable requirement of 100 msec latency 
maximum for an XVS is suggested. 

This same latency test method was used to diagnose the prominent latency sources in our 
proof-of-concept XVS. 

• First, four parallel video streams were reduced to a single video input. The video 
capture process was still used as our symbology generation method. With this 
set-up, the latency was reduced to 230 msec. This result indicates that there was 
approximately 100 msec of latency associated with the overhead of running 4 
parallel capture cards and displaying this Quad-HD video on the T22 1 monitor. 

• Next, the Quad-HD monitor was replaced with a commercial off-the-shelf 
Samsung (Model LN32A550) HDTV monitor, running 1920x1080 resolution. 
With this set-up the latency was reduced to 200 msec. This result indicates there 
was only 30 msec of additional latency associated with the T221 monitor 
compared to a present-day liquid crystal HDTV monitor. 

• Finally, the video capture cards were bypassed and the HDTV video cameras 
directly drove the Samsung HDTV monitor, running 1920x1080 resolution. With 
this set-up the latency was reduced to 155 msec. This result indicates that 
approximately 45 msec of latency is associated with the frame grabbing process. 
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Laptop with Timer Display 


Quad-HQ Image of Laptop Timer 


Figure 17: XVS Proof-of-Concept Quad-HD End-to-End Latency Test 


Detailed investigations into the exact processes that are causing these latencies have not 
yet been undertaken. The data suggest that two significant factors are influencing these 
results. 

First, the HDTV cameras being used are not providing a sufficient real-time video source. 
The latency for just the camera and HDTV monitor is 155 msec. Even if the camera was 
running as slow as a 24 Hz frame rate, the minimum expected latency would be on the 
order of 41.67 msec. Since the measured latency is so much greater than this minimum, 
it indicates that there are significant other factors included in the camera system (e.g., 
frame buffering, sensor element integration and processing, etc.) that degrade real-time 
performance. 

In addition, the T221 monitor used in this proof-of-concept employs a limited refresh 
rate. While the data does not indicate that the monitor is a major latency source, the new 
Quad-HD monitors expected in the near future will have significantly improved video 
refresh rates and video processing circuitry which should reduce the latency contribution. 

Before diagnosing and creating remedies for this XVS proof-of-concept, better cameras 
and monitors should be procured. Once this happens, the hardware and software 
processes which control the video frame grabbers will have to be addressed to overcome 
the 100+ msec of latency that they contribute. 
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8. Concluding Remarks 

This report provides the background for conceptual XYS design standards for display and 
sensor resolution. 

The resolution requirements were derived from the basis of equivalent safety and 
performance to forward- facing windows currently used and certificated for 
“conventional” aircraft. Three measures for equivalent safety and performance were 
investigated: a) human vision performance equivalence; b) see-and-avoid performance 
and safety; and c) see-to-follow performance. 

From these three factors, a minimum but perhaps not sufficient resolution requirement of 
60 pixels per degree was shown for human vision equivalence. However, see-and-avoid 
and see-to-follow performance requirements are nearly double. These results need to be 
verified and validated by in-flight testing. 

XVS display and sensor resolution requirements should be developed using modulation 
transfer function-based criteria instead of simple pixel per degree measures. The MTF 
measures would more accurately capture the true system performance and more 
accurately reflect in-flight performance. 

This report also reviewed historical XVS testing. The trends in co mm ercial video 
systems are very favorable such that emerging technology should be available to meet the 
needs of an XVS system. Proof-of-concept testing has identified one significant 
limitation however. The latency of the XVS can be a significant technical hurdle. 
Methods for co mm ercial off the shelf architectures are being explored, but custom-video 
processing may be needed instead. Fortunately, custom video work is no longer a 
technological hurdle, but merely an engineering task. This work should be done to 
increase the Technology Readiness Levels for XVS technology so the lack of forward- 
facing windows will not be a risk- factor in the successful launch of commercial and 
business supersonic aircraft. 
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Appendix A 

Recommended XVS Flight Test and Evaluation Maneuvers 


The following maneuvers are a subset of evaluation tests reco mm ended for the research, 
development, test and evaluation of XVS and other “visionic” systems. The term 
“visionics” is used as the name for a generic device which provides a pictorial 
representation of the external scene (i.e., the environment in which the aircraft is flying), 
generated by electronic means such as electro-optical sensors, radars and/or a database 
(synthetic vision). In this case, the XVS (or “visionic” system) is used to replicate the 
functionality of a forward-facing window for see-and-avoid and see-to-follow. 

These maneuvers are a subset of those which have been previously used during the HSR 
program in the initial development of an XVS (Yang et al, 2000, Gandhi et al, 2003) and 
are also being used for “sense-and-avoid” testing for UAVs (Shakemia et al , 2007). 

For each of the test maneuvers that follow, some brief operational background and 
specific test objectives is given. The test scenario is then described. Extensions and 
modifications to these maneuvers can be easily made to accommodate different 
operational contexts. Some of these are briefly discussed in the “operational 
background” for each task. 

The general term - “visionics” - is used in an attempt to remain technology agnostic. 

“See-and- Avoid” - Visual Target Detection 

1. Operational Background 

Visionics devices may provide visual flight reference information sufficient to allow 
operational capabilities enabling equivalent Visual Flight Rules operations; that is, the 
ability to see-and-avoid other aircraft. The ability of the pilot (operator) to see other 
aircraft (i.e., “targets”) is critical for the purposes of self-separation and see-to-follow. 

Comparative testing is used to quantify the ability of the pilot, when using a visionics 
device, to meet this see-and-avoid / see-to-follow requirement. Comparative testing is 
necessary since absolute criteria are not available. 

2. Test Objectives 

The test objective is to measure the ability of the visionics device to provide the visual 
cues to enable the evaluation pilot to detect, recognize, and identify other aircraft traffic. 
Pure “expansion” (i.e., the target is represented by an increasing number of pixels without 
a translation or change in the position of the target on the display) of a target is typically 
tested as it represents the most difficult recognition task. 

3. Test Scenario Definition 
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The following maneuver highlights one of numerous verification tests which may be used 
for comparative evaluation against laboratory or simulation tests for in-flight 
determination of traffic (target) detection, recognition, and identification (Figure A-l). 
This test emphasizes visual traffic (i.e., “target”) detection by the evaluation pilot using 
“expansion” along a horizontal trajectory away from the evaluation pilot. This maneuver 
should also be conducted in the reverse direction (i.e., ownship overtaking the target) to 
test for “expectancy” effects on the part of the evaluation pilot. 

Other trajectories, across path, along path, with and without vertical trajectories, may also 
be performed to evaluate (in-flight) visionics performance and operator detection (of 
“just noticeable differences”) such as Pure Expansion with Pitch Trajectories, False 
Expansion, Pure Translation Expansion with Pitch, and Expanding Background 
Expansion with Translation. 


Time - 1 


Vi 



Time - 2 



Figure A-l: Visual Target Detection - Expansion With Horizontal Trajectories 


The intent of this scenario is to evaluate the visual (acuity and contrast) performance of 
the visionics system in-flight for target detection, recognition and identification. The 
target aircraft (labeled “Target” in Figure A-l) presents a traffic situation to the 
evaluation pilot who is flying ownship aircraft: 

• The task begins with the target aircraft flying in loose formation to the left or 
right and behind ownship. 

• When the evaluation pilot with the visionics device is ready, the pilot will radio 
the target aircraft to move into the task starting position ahead of ownship on the 
same heading. 

• The target aircraft begins the task at the same speed, altitude and heading of 
ownship and positioned nominally 100 feet ahead (Time 1 in Figure A-l). 
Ownship maintains constant altitude, speed, and heading. 

• On a mark from the evaluation pilot (i.e., data “event marker” triggered and radio 
transmission of "Start Maneuver"), the target aircraft accelerates to a constant 
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airspeed and pulls away from ownship (Time 2), maintaining heading and 
altitude. 

• The first test-point occurs when the target is no longer discernible to the 
evaluation pilot using the visionics device. 

• If comparative testing can be done in real-time, a second test-point occurs when 
the target is no longer discernible to another observer using natural vision. 

Weather, atmospheric obscurants and conditions, and time-of-day variations may 
significantly affect the results of these tests; thus, measurements of these conditions 
should be made, as possible and practical, and repeated measurements taken for statistical 
reliability. The results will also be influenced by the observers’ visual acuity so 
individual testing should be made and numerous observers used. 

Important performance metrics are as follows: 

• Aircraft separation distance when target aircraft is no longer visible, with and 
without the visionics device. 

• Comparison against “theoretical” visual target recognition, and identification 
criteria. 


“See-and-Avoid” - Departure Level-off 

1. Operational Background 

The following maneuver is analogous to the “See-and-Avoid - Visual target detection” 
task with the notable exception that ground clutter is a significant component to the target 
identification and recognition. 

This task is also critical in that it is an all-too-often operational scenario. In this case, the 
“ownship” aircraft is being vectored toward an airport and the target aircraft is departing 
on opposite headings. The target aircraft on a departure climb-out errs and flies through 
an Air Traffic Control “level-off’ altitude with ownship flying a constant altitude. This 
maneuver tests the ability of the visionics system to support pilot detection of the target 
aircraft blunder. 

2. Test Objectives 

The test objective is to measure the ability of the visionics device to provide the visual 
cues to enable the pilot to identify other aircraft traffic, follow that traffic, and maintain 
separation in the landing pattern in the presence of ground clutter. 

3. Test Scenario Definition 

The following maneuver highlights one of numerous verification tests which may be used 
for comparative evaluation against laboratory or simulation tests for in-flight 
determination of traffic (target) detection, recognition, and identification (Figure A-2). 
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This task emphasizes visual traffic (i.e., “target”) detection by the evaluation pilot using 
“expansion” toward the evaluation pilot in ground clutter. This maneuver should also be 
conducted in the reverse direction (i.e , target aircraft flying away from ownship) to test 
for “expectancy” effects on the part of the evaluation pilot. 

Other trajectories, across path, along path, with and without vertical trajectories, may also 
be performed to evaluate (in-flight) the visionics performance and operator detection (of 
“just noticeable differences”) such as Pure Expansion with Pitch Trajectories, False 
Expansion, Pure Translation Expansion with Pitch, and Expanding Background 
Expansion with Translation but these are lower priority. 





Figure A-2: Visual Target Detection - Departure Level-Off 


The intent of this scenario is to evaluate the visual (acuity and contrast) performance of 
the visionics system in-flight for target detection, recognition and identification. The 
target aircraft (labeled B-200 in Figure A-2) presents a traffic situation to the evaluation 
pilot who is flying ownship aircraft: 
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• The task begins with the target aircraft (labeled “B-200” in Figure A-2) and 
ownship (labeled “TIFS” in Figure A-2) flying in opposite directions, separated in 
altitude (labeled “Time 1”). 

• When the evaluation pilot with the visionics device is ready, the pilot will radio 
the target aircraft to begin the task. 

• The target aircraft begins the task at the same speed but opposite heading of 
ownship. The altitude separation is based on geometry (as follows) and flight test 
considerations. 

• The target aircraft begins a climb (“Time 2” in Figure A-2) to achieve a constant 
angle climb as seen by ownship. Airspeed and climb rate are determined by flight 
test considerations to maintain this constant angle on the visionics device 
(“Expansion” testing without target translation). 

• The target aircraft levels-off the climb to maintain an altitude buffer as required 
for safety of flight. 

• The first test-point occurs when the target is detected (and/or recognized, 
identified) by the evaluation pilot using the visionics device. 

• If comparative testing can be done in real-time, a second test-point occurs when 
the target is detected (and/or recognized, identified) by another observer using 
natural vision. 

Weather, atmospheric obscurants and conditions, and time-of-day variations may 
significantly affect the results of these tests; thus, measurements of these conditions 
should be made, as possible and practical, and repeated measurements taken for statistical 
reliability. The results will also be influenced by the observers visual acuity so individual 
testing should be made and numerous observers used. 

Important performance metrics are as follows: 

• Aircraft separation distance when target aircraft is no longer visible, with and 
without the visionics device. 

• Comparison against “theoretical” visual target recognition, and identification 
criteria. 

See-to-Follow Performance - Self-Separation and Flight-Following 

1. Operational Background 

The following outlines a methodology - one of several - for evaluation (validation) of 
visionics devices as a way to evaluate equivalent visual operations, in this case, for “see- 
to-follow” capabilities. 

2. Test Objectives 

The test objective is to measure the ability of the visionics device to provide the visual 
cues to enable the pilot to identify other aircraft traffic, follow that traffic, and maintain 
separation in the landing pattern. 
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The following maneuver highlights one of numerous “typical” YFR operational problems 
associated with traffic following and self-separation (Figure A-3). This one emphasizing 
traffic positions inboard of the evaluation pilot who in this case is seated in the left-hand 
seat. 

3. Tesj Scenario Definition 



Figure A-3: See-to-Follow Equivalent Visual Operation 


The intent of this scenario is to have the target aircraft (labeled “Target” in Figure A-3) 
present a traffic situation to the evaluation pilot who is flying ownship aircraft during the 
entry of the downwind leg of a nominal approach to landing which requires close 
monitoring of the incoming traffic to ensure adequate separation. The evaluation pilot 
must identify and observe the target aircraft’s movement to enter the traffic pattern 
behind it. 

• The ownship aircraft begins a circling holding pattern in a loiter area abeam the 
downwind leg. 

• The target aircraft (“Target”) begins on a downwind leg of a right-hand traffic 
pattern, at 1500 ft above field level (AFL), holding a constant approach airspeed. 

• The evaluation pilot will maneuver the ownship aircraft to maintain a constant in- 
trail spacing (e.g., 3 nm, or 5 nm) spacing on the target aircraft and follow it to 
landing (Time 2). 

• On the base turn, the target will descend to 1000 ft AFL. The target will hold this 
altitude until glideslope intercept. Ownship will maintain 1500 ft AFL until 
glideslope intercept. 
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• Both aircraft continue the landing approach through Time 3 and Time 4. The 
target aircraft will execute a go-around at 200 ft AFL, ending the evaluation. 

• This same scenario may also be conducted for a left-hand approach, emphasizing 
traffic positions outboard of the evaluation pilot who is seated in the left-hand 
seat. 

The test data should reflect when the target is detected, recognized, and identified by the 
evaluation pilot using the visionics device. If comparative testing can be done in real- 
time, second test-points occur when the target is detected, recognized, and identified by 
another observer using natural vision. The ability to maintain separation (spacing) should 
be recorded. Different targets also provided different levels of visionics performance 
testing. 

Weather, atmospheric obscurants and conditions, and time-of-day variations may 
significantly affect the results of these tests; thus, measurements of these conditions 
should be made and repeated measurements taken for statistical reliability. The results 
will also be influenced by the observers visual acuity so individual testing should be 
made and numerous observers used. 

See-and- Avoid / See-to-Follow Performance - Parallel Runway Ops 

1. Operational Background 

The following outlines a methodology - one of several - for evaluation of visionics 
devices as a way to evaluate equivalent visual operations, in this case, for “see-to-follow” 
capabilities during parallel runway operations. 

2. Test Objectives 

The test objective is to measure the ability of the visionics device to provide the visual 
cues to enable the pilot to identify other aircraft traffic, follow that traffic, and maintain 
separation in the landing pattern. 

The following maneuver highlights one of numerous “typical” VFR operational problems 
associated with see-and-avoid and see-to-follow in YFR operations, in this case, during 
parallel runway operations (Figure A-4). This one emphasizes traffic positions forward 
and inboard of the evaluation pilot who is seated in the left-hand seat (until the final 
approach). 

3. Test Scenario Definition 

The intent of this scenario is to have the target aircraft (labeled “Target” in Figure A-4) 
present a traffic situation to the evaluation pilot who is flying ownship aircraft during 
opposite traffic patterns, serving parallel runway approach and landings. The task 
requires close monitoring of the opposite direction traffic to ensure adequate separation. 
The evaluation pilot must identify and observe the target aircraft to maintain safe 
separation and monitor for potential “blunders.” 
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Time 1 


Target 



Figure A-4: See-and- Avoid / See-to-Follow Performance - Parallel Runway Ops 


• Both the target and ownship aircraft begin on downwind legs to parallel runways 
using left- and right-hand patterns, respectively. The target aircraft ownship can 
be a “virtual runway” to ease operational constraints (e.g., by using waypoints for 
vectors). The spacings between the parallel runways should be varied to test 
visionics performance. 

• The size and aircraft spacings in the traffic pattern are designed based on flight 
test considerations, as follows. 

• The target aircraft (“Target”) begins on a downwind leg ahead of ownship in the 
pattern (“Time 1”) so that: a) some reasonable amount of time is provided to the 
evaluation pilot flying ownship to visually acquire the target while on opposite 
facing base legs (“Time 2”); and, b) if executed in a timely fashion, the ownship 
and target aircraft will have 2 nm, 3 nm or 5 nm spacings (experimentally varied) 
on final to their respective runways (“Time 3”). 

• The evaluation pilot will maneuver the ownship aircraft to maintain a constant in- 
trail spacing (e.g., 2 nm, 3 nm, or 5 nm) spacing on the target aircraft and follow it 
to landing (“Time 4”). 

• A test for awareness may be experimentally varied by having the target aircraft, 
unbeknownst to the evaluation pilot flying ownship, fly to the wrong parallel 
runway (i.e., incurring onto ownship’ s runway). 

• This same scenario may also be conducted for a right-hand approach by ownship. 
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The test data should reflect when the target is detected, recognized, and identified by the 
evaluation pilot using the visionics device. If comparative testing can be done in real- 
time, second test-points occur when the target is detected, recognized, and identified by 
another observer using natural vision. The ability to maintain separation (spacing) should 
be recorded. Finally, separation spacing and runway spacing can be varied to assess 
visual acuity and contrast performance. Different targets should also be used to test 
different levels of visionics performance. 

Weather, atmospheric obscurants and conditions, and time-of-day variations may 
significantly affect the results of these tests; thus, measurements of these conditions 
should be made and repeated measurements taken for statistical reliability. The results 
will also be influenced by the observers visual acuity so individual testing should be 
made and numerous observers used. 
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